feat(sftp): DDS integration into the rover (DHEI-20905) - #445
feat(sftp): DDS integration into the rover (DHEI-20905)#445valenok4000 wants to merge 35 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR extends the Rover-facing OpenAPI contract to support configuring a DDS-based file-transfer backend (file exposures/subscriptions), regenerates the Go API types from the updated spec, and introduces a feature flag to keep the file domain disabled until the downstream domains are available.
Changes:
- Added
FileExposure,FileSubscription, andPublicKeyschemas to the OpenAPI spec and wired them into theExposure/Subscriptiondiscriminator unions. - Regenerated
oapi-codegenGo types to include the new file exposure/subscription variants and union helper methods. - Added a new
FeatureFilefeature flag (default disabled) to gate upcoming file/DDS domain functionality.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| rover-server/internal/api/server.gen.go | Regenerated OpenAPI Go models/unions to include file exposure/subscription variants. |
| rover-server/api/openapi.yaml | Adds file exposure/subscription schemas and updates discriminator mappings for request unions. |
| common/pkg/config/feature.go | Adds a new FeatureFile flag (disabled by default) for the file/DDS domain rollout. |
Files not reviewed (1)
- rover-server/internal/api/server.gen.go: Generated file
Comments suppressed due to low confidence (1)
rover-server/internal/api/server.gen.go:4
- The generated server stub no longer contains the SPDX license header. This can break REUSE/SPDX compliance checks for code files; if this file is intentionally generated, the generator/template should still emit the header (or the repo should explicitly exclude this path).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| Exposure: | ||
| type: object | ||
| discriminator: | ||
| propertyName: type | ||
| mapping: | ||
| api: '#/components/schemas/ApiExposure' | ||
| event: '#/components/schemas/EventExposure' | ||
| file: '#/components/schemas/FileExposure' | ||
| oneOf: | ||
| - $ref: '#/components/schemas/ApiExposure' | ||
| - $ref: '#/components/schemas/EventExposure' | ||
| - $ref: '#/components/schemas/FileExposure' |
| Subscription: | ||
| type: object | ||
| discriminator: | ||
| propertyName: type | ||
| mapping: | ||
| api: '#/components/schemas/ApiSubscription' | ||
| event: '#/components/schemas/EventSubscription' | ||
| file: '#/components/schemas/FileSubscription' | ||
| oneOf: | ||
| - $ref: '#/components/schemas/ApiSubscription' | ||
| - $ref: '#/components/schemas/EventSubscription' | ||
| - $ref: '#/components/schemas/FileSubscription' |
add mapping for exp/subs into the rover
change logic on client creation
ron96g
left a comment
There was a problem hiding this comment.
I understand that this is not done yet but here is a first review of the current code.
| type FileExposureSpec struct { | ||
| // Approval configures how subscriptions to this file type are approved. | ||
| // +kubebuilder:default=Simple | ||
| Approval ApprovalStrategy `json:"approval,omitempty"` |
There was a problem hiding this comment.
Must be inside of an Object with strategy: ApprovalStrategy and TrustedTeams: []string, see API and Event
| // FileTypeRef references the file-domain FileType this exposure belongs to. | ||
| // It must live in the same namespace as this FileExposure. | ||
| // +kubebuilder:validation:Required | ||
| FileTypeRef ctypes.ObjectRef `json:"fileTypeRef"` |
There was a problem hiding this comment.
The ref on this layer is done via the string-value of fileType and not via ObjectRef. This is needed as we might need to switch the active FileType (if migrated)
| // ZoneServiceConfigRef references the zone-scoped service configuration | ||
| // (provided by the SFTP/DDS domain) used to reach the backend for this exposure. | ||
| // +optional | ||
| ZoneServiceConfigRef *ctypes.ObjectRef `json:"zoneServiceConfigRef,omitempty"` |
There was a problem hiding this comment.
On this Layer we just pass the Zone-Ref and the domain itself must resolve it to its config, so Zone ctypes.ObjectRef
| type FileSubscriptionSpec struct { | ||
| // FileTypeRef references the file-domain FileType this subscription belongs to. | ||
| // +kubebuilder:validation:Required | ||
| FileTypeRef ctypes.ObjectRef `json:"fileTypeRef"` |
There was a problem hiding this comment.
Same as for the Exposure. Ref via string-value and not via Ref
| Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"` | ||
|
|
||
| // Active indicates whether this subscription has been provisioned. | ||
| Active bool `json:"active,omitempty"` |
There was a problem hiding this comment.
Active flag is only for Exposure and Type, not subscription. There we use conditions
| // file domain from a rover-domain FileSpecification (1:1) and is the canonical | ||
| // resource that FileExposure (1:1) and FileSubscription (1:n) reference via their | ||
| // fileTypeRef (mirrors event.EventType). | ||
| type FileTypeSpec struct { |
There was a problem hiding this comment.
The FileType string is missing (similiar to eventType and basePath). Do we also need to add a specification field with the file-manager ref?
There was a problem hiding this comment.
add Type as it's done for event. add specification
|
|
||
| // ExposureRef references the file-domain FileExposure created for this file type (1:1). | ||
| // +optional | ||
| ExposureRef *ctypes.ObjectRef `json:"exposureRef,omitempty"` |
There was a problem hiding this comment.
Both ExposureRef and SubscriptionRefs break our pattern and hierarchy: Type > Exposure > Subscription.
The FileType does not know about its Exposure and Subscription
| // +optional | ||
| Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"` | ||
|
|
||
| // Active indicates whether this FileType has been provisioned. |
There was a problem hiding this comment.
Comment is a bit misleading. This active-flag is used to indicate that THIS CR is the active one (singleton)
| type: array | ||
| items: | ||
| $ref: '#/components/schemas/EventScope' | ||
| FileExposure: |
There was a problem hiding this comment.
This is aligned with the current Openapi-Spec?
add / change file/rover api add feature config, handler controller wh for file FileSpecification in progress
ron96g
left a comment
There was a problem hiding this comment.
See comments. The whole handler-logic in the file domain is still missing, correct?
| // PublicKeys are the SSH public keys registered for the provider's SFTP user. | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:validation:MinItems=1 | ||
| PublicKeys []PublicKey `json:"publicKeys"` |
There was a problem hiding this comment.
Why is this a root-level field? Did we not say that we want sub-objects like
sftp:
publicKeys: []PublicKey
cloudwalker:
some: stuff
**Cloudwalker is just a placeholder here :D
There was a problem hiding this comment.
spec:
approval: SIMPLE
fileType: ...
sftp:
publicKeys:
- key: ssh-rsa AAAAB3NzaC1...
label: coran-int-dds
visibility: ENTERPRISE
zone:
name: ...
namespace: ...
| // PublicKeys are the SSH public keys registered for the consumer's SFTP user. | ||
| // +kubebuilder:validation:Required | ||
| // +kubebuilder:validation:MinItems=1 | ||
| PublicKeys []PublicKey `json:"publicKeys"` |
There was a problem hiding this comment.
adjusted to the structure (from design)
spec:
sftp:
clientId: <subscription application....>
publicKeys:
- key: ssh-rsa AAAAB3NzaC1...
label: coran-int-dds
- key: ssh-rsa AAAAB3NzaC1...
label: coran-int-dds-2
|
|
||
| out.Event = mapEventExposure(eventExp) | ||
|
|
||
| case "file": |
There was a problem hiding this comment.
Is there not "out" mapping for this? What about applicationinfo?
There was a problem hiding this comment.
I add out mapping.
applicationInfo???
| import ( | ||
| . "github.com/onsi/ginkgo/v2" | ||
| . "github.com/onsi/gomega" | ||
| . "github.com/onsi/gomega/gstruct" |
There was a problem hiding this comment.
deleted (revert to initial state, it was (over)generated by AI
| // It will be called by the file-domain handler | ||
| // (rover/internal/handler/rover/file, added in DHEI-20903) when logging/serialising | ||
| // the selected variant while creating the file-domain CRD. | ||
| func (v FileVariant) String() string { |
There was a problem hiding this comment.
We dont need this. We no longer support multiple variants
There was a problem hiding this comment.
that's good, will delete it
|
|
||
| fileSubscription.Spec = filev1.FileSubscriptionSpec{ | ||
| FileType: sub.FileType, | ||
| PublicKeys: mapPublicKeys(sub.PublicKeys), |
There was a problem hiding this comment.
I would add zone here as well just to keep it similiar. Even though it not used right now, correct?
There was a problem hiding this comment.
Also I think you will need the Requestor values for the Approval-Handling
There was a problem hiding this comment.
I would add zone here as well just to keep it similiar. Even though it not used right now, correct?
Refering to the internal documentation and design https://gitlab.devops.telekom.de/dhei/products/controlplane/docs/dds-file-integration
we don't have this zone for specification.
There was a problem hiding this comment.
I would add zone here as well just to keep it similiar. Even though it not used right now, correct?
We keep it like this in design, not sure that similarity must be our point of truth.. will see
| // MakeName generates a deterministic resource name for a file exposure or | ||
| // subscription: "<fileType>--<owner>" (spec_dcp naming), normalized. | ||
| func MakeName(fileType, ownerName string) string { | ||
| return filev1.MakeFileTypeName(fileType) + "--" + ownerName |
There was a problem hiding this comment.
There are functions to Normalize these values, see labelutil
| // The file type identifier lives in metadata.name (no spec.type field in the | ||
| // internal CRD, per spec_dcp); the client-side name==type rule is enforced by | ||
| // rover-server / roverctl. | ||
| if st := filespecification.Spec.StorageType; st != "" && st != roverv1.FileStorageTypeSFTP { |
There was a problem hiding this comment.
Not needed as done via CEL. Also I would remove it as not needed for now. We only have sftp and no plans to add more
|
|
||
| // +kubebuilder:webhook:path=/validate-rover-cp-ei-telekom-de-v1-filespecification,mutating=false,failurePolicy=fail,sideEffects=None,groups=rover.cp.ei.telekom.de,resources=filespecifications,verbs=create;update,versions=v1,name=vfilespecification-v1.kb.io,admissionReviewVersions=v1 | ||
|
|
||
| type FileSpecificationCustomValidator struct { |
There was a problem hiding this comment.
I think the entire webhook is not needed
| exposesFiles := slices.ContainsFunc(rover.Spec.Exposures, func(exp roverv1.Exposure) bool { | ||
| return exp.Type() == roverv1.TypeFile | ||
| }) | ||
| if (subscribesToFiles || exposesFiles) && !roverv1.IsFileTypeZoneSupported(rover.Spec.Zone) { |
There was a problem hiding this comment.
As other comments states. Should be done via feature-flag logic in admin and not hardcoded in rover.
… and fix the rover mapper out remove zone_supported flag and webhook as it's covered by featureEnabled flag add rbac for file domain
…struct" lib as it was added by AI add crypto lib and ssh pub key validation
move func desc to the original func
# Conflicts: # common/pkg/config/feature.go # rover-ctl/pkg/handlers/v0/rover.go # rover-ctl/pkg/handlers/v0/rover_test.go # rover-server/api/openapi.yaml # rover-server/internal/api/server.gen.go # rover-server/internal/mapper/rover/in/__snapshots__/exposure_test.snap # rover-server/internal/mapper/rover/in/__snapshots__/rover_test.snap # rover-server/internal/mapper/rover/in/__snapshots__/subscription_test.snap # rover-server/internal/mapper/rover/in/exposure.go # rover-server/internal/mapper/rover/in/subscription.go # rover-server/internal/mapper/rover/out/exposure.go # rover/api/v1/rover_types.go # rover/api/v1/zz_generated.deepcopy.go # rover/cmd/main.go # rover/config/crd/bases/rover.cp.ei.telekom.de_rovers.yaml # rover/config/rbac/role.yaml # rover/go.mod # rover/internal/controller/rover_controller.go # rover/internal/handler/rover/application/application.go # rover/internal/handler/rover/handler.go # rover/internal/webhook/v1/rover_webhook.go
# Conflicts: # rover-server/internal/mapper/rover/in/__snapshots__/exposure_test.snap # rover-server/internal/mapper/rover/in/__snapshots__/rover_test.snap # rover-server/internal/mapper/rover/in/__snapshots__/subscription_test.snap # rover-server/internal/mapper/rover/out/exposure.go # rover/api/v1/rover_types.go # rover/api/v1/zz_generated.deepcopy.go # rover/internal/handler/rover/handler.go
fix some issues
# Conflicts: # rover-server/internal/api/server.gen.go
Make possible the DDS configuration from the Rover.